Many modern network adapters are configured to include multiple transmit queues. Typically, multiple transmit queues are made available in order to provide enhanced quality of service (“QoS”), the preferential treatment of some packets over others, by providing queues of varying priority levels (e.g., one queue per priority level), into which packets may be deposited while awaiting transmission on a network. Many network communication protocols, such as Internet Protocol, version 6 (“IPv6”) (IETF draft standard RFC 2460, December 1998, “Internet Protocol, Version 6 (IPv6) Specification”) provide a capability to specify a priority for the packet as a part of an IP header. Providing queues of varying priority levels enables the network adapter to transmit higher priority packets before lower priority packets, even though the higher priority packets may have been queued subsequent to the lower priority packets.
Because packets within the same flow (a single communication may comprise one or multiple packets depending on the size of the communication) will be designated with the same priority, providing one queue per priority level ensures that packets will be sent in the same order in which they were processed by a protocol stack, thereby eliminating, to the extent possible, the potential that the packets will be received out-of-order at the destination node on the network. While not an issue in regard to all communication protocols, the order in which packets are transmitted and received can represent a problem for some communication protocols and particularly one widely used communication protocol, Transmission Control Protocol (“TCP”) (IETF standard RFC 793, Sep. 1, 1981, “Transmission Control Protocol”). In one implementation of TCP for example, if a packet is received out-of-order (i.e., the order in which it was processed by the protocol stack) by more than a small margin (e.g., a swap with an adjacent packet), it will be considered lost, and will have to be retransmitted. Providing one queue per one or more priority levels helps to alleviate this problem because all packets of equal priority will be deposited in the same queue. However, in network environments in which a protocol that does not include a priority indicating feature is in use, or the protocol in use supports fewer levels of priority than there are queues available with a network adapter, at least a portion of the multiple transmit queues associated with the network adapter remain unused.
The reader will appreciate that a packet comprises a package of information transmitted as a single unit from a source node on a network to a destination node on the network, and typically includes data, as well as addressing, routability, and reliability information in the form of headers and/or footers to enable the network's communication protocols to deliver the data within the packet to the intended recipient, and to ensure that the data arrives intact and without corruption. As transmit requests are generated from applications in response to user input or otherwise, the transmit requests are transferred to a protocol stack, such as a Transmission Control Protocol/Internet Protocol (“TCP/IP”) protocol stack for encapsulation into the packet. After incorporating the data and other information into the packet, the packet is transferred to a device driver associated with a network adapter to be placed in a queue, awaiting transmission on the network. The device driver generally comprises a software component that permits a computer system to communicate with a device (e.g., a network adapter), and manipulates data in order to transmit data to the device (e.g., depositing packets in a queue and informing the network adapter that packets are waiting to be transmitted).
In a multiprocessor system, it is possible to have multiple transmit requests arrive at a “send packet” function of the device driver simultaneously on different processors. As mentioned previously, if the packets of the transmit requests are of equal priority, different priorities that are sharing a queue, or a priority-indicating protocol is not being used, then the packets will be deposited in the same queue via current queuing mechanisms to await transmission on the network. In order to provide access to the “send packet” function resources associated with a particular queue, the device driver may use semaphores to indicate to potential users that the “send packet” function resources associated with the particular queue are in use, thereby preventing access by more than one thread.
In a Windows® operating system environment for example, the semaphores are referred to as “spinlocks.” Spinlocks guarantee that only one process at a time will be given access to a particular resource, thereby ensuring data integrity within the resource. As a consequence of the foregoing, when a second processor, for example, attempts to acquire a resource (e.g., a queue specific element of the “send packet” function) that is already being used by a first processor, for example, the second processor “spins,” and waits for the spinlock to be released. While waiting for the spinlock to be released, the second processor remains in a “busy wait,” and is not free to perform other tasks. Often, in network environments designed for high throughput levels (e.g., gigabit or 10 gigabit connections), the throughput level of the network connection itself may be limited by processor bandwidth, particularly, cycles spent preparing and processing network transmit requests.